Information recording medium, device and method for playing back 3d images

ABSTRACT

Provided is a recording medium allowing random access to be performed in playing back 3D graphics. The recording medium comprises a digital stream area where a digital stream including temporally-arranged GOP pairs is recorded; and a map information area where map information is recorded, the map information indicating entry addresses in one to one correspondence with entry times on a time axis of the digital stream, each entry address showing a beginning of a corresponding GOP pair region in the digital stream area, wherein each GOP pair including first-type and second-type GOPs, each first-type GOP is data indicating a plain view picture set to be played back from a corresponding entry time, and each second-type GOP is data to be played back with a corresponding first-type GOP to provide a stereoscopic view, the data indicating a difference between a stereoscopic picture set and the plain view picture set.

BACKGROUND OF THE INVENTION

(1) Field of the Invention

The present invention relates to an application format when 3D graphicsis recorded on a recording medium.

(2) Description of the Related Art

In recent years, the next generation DVD standards called the Blu-raydisc and HD DVD have been established, and therefore high definition andhigh-quality sound optical discs have been common among users.

Regarding quality of moving images that can be recorded on such opticaldiscs, while conventional DVDs are SD (Standard Definition), Blu-raydiscs are HD (High Definition) with resolutions up to 1920×1080, andtherefore can store therein images having higher image quality.

In recent years, the number of visitors to movie theaters has beendecreasing with the expansion of package media such as DVDs. Therefore,in the U.S. and in Japan, establishment of movie theaters in whichvisitors can enjoy three dimensional movies (3D movies) have beenencouraged in order to increase the number of visitors to movietheaters. One of the factors that contributes to such changes in themovie theaters is that an environment has been developed in which 3Dgraphics can be generated comparatively easily since numerous recentmovies have been made using CG (computer graphics).

In view of such a background, 3D content recorded on the above-mentionednext generation DVDs such as Blu-ray discs and HD DVDs will be desiredby many users.

SUMMARY OF THE INVENTION

Although being capable of simply playing back 3D content from beginningto end is enough at the movie theaters, such playback is not enough inorder to introduce the 3D technology to homes in a form of an opticaldisc. That is, it is necessary to ensure that random access and the likecan be performed as before in terms of usability for users whenintroducing the 3D technology to homes. As a method of realizing 3Dviewing at home, a method of using parallax video (composed of two videostreams that are based on binocular parallax) may be taken intoconsideration. In that case, it is problematic how to synchronize thetwo video streams at the time of random access in order to ensure thatrandom access can be performed. If the two video streams cannot besynchronized, a time period might occur in which one video stream can beproperly decoded while the other video stream cannot be decoded when theuser plays back from an arbitrary time point at which random access canbe performed. This is because data necessary for playback is notprovided to a decoder. As a result, stereoscopic viewing using parallaxvideo cannot be performed in such time period.

The present invention has an objective to provide a recording mediumwith which random access can be reliably performed in playing back 3Dgraphics.

In order to achieve the above objective, one aspect of the presentinvention is a recording medium comprising: a digital stream area inwhich a digital stream is recorded, the digital stream including aplurality of temporally-arranged GOP (group of pictures) pairs; and amap information area in which map information is recorded, the mapinformation indicating entry addresses in one to one correspondence withentry times on a time axis of the digital stream, each of the entryaddresses showing a beginning of a corresponding one of GOP pair regionsin the digital stream area, wherein each of the GOP pairs is composed ofa first-type GOP and a second-type GOP, each first-type GOP is dataindicating a set of plain view pictures to be played back from acorresponding one of the entry times, and each second-type GOP is datato be played back together with a corresponding one of the first-typeGOPs to provide a user with a stereoscopic view of the digital stream,the data indicating a difference between a set of stereoscopic picturesand the set of plain view pictures.

With the above-stated structure, since the pairs of the first group ofpictures and the second group of pictures exists in the GOP pair regionsindicated by the entry addresses in the recording area of the digitalstream that correspond to the entry times, it is possible to reliablyallow a user to perform stereoscopic viewing of moving pictures evenwhen playback starts at an arbitrary entry time on the time axis of thedigital stream. Therefore, the user can easily enjoy viewing 3D graphicsat home.

Each of the first-type GOPs and each of the second-type GOPs may bedivided into a first-type set of packets and a second-type set ofpackets, respectively, the first-type sets of packets and thesecond-typesets of packets may be multiplexed together, all of thepackets being assigned with consecutive packet numbers according to anorder of multiplexing, a header packet from among each of the first-typesets of packets may precede, on the digital stream, a header packet fromamong a corresponding one of the second-type sets of packets, and eachof the entry addresses in the map information may be represented as oneof the packet numbers assigned to a header packet from among acorresponding one of the first-type sets of packets.

The entry addresses each showing a header packet of a correspondingfirst-type GOP are in one to one correspondence with the entry times.Therefore, by reading packets from the recording medium in accordancewith the entry addresses, it is possible to reliably send the firstgroup of pictures to a video decoder without sending unnecessary data.Therefore, it is possible to immediately provide stereoscopic viewingstarting from an entry point desired by the user when random access isperformed on a video stream.

Each of the second-type sets of packets divided from a corresponding oneof the second-type GOPs may be located before a next one of the entryaddresses that is immediately next to one of the entry addresses thatrelates to a corresponding one of the first-type GOPs.

It is ensured that a complete pair of the first group of pictures andthe second group of pictures can be sent to the video decoder by readingpackets from a packet (m) indicated by an entry address (i) to a packet(n−1) immediately preceding a packet (n) indicated by an entry address(i+1). Since it is ensured, in the case of performing random access withreference to the map information, that the complete pair of the firstgroup of pictures and the second group of pictures which can providestereoscopic viewing are sent to the video decoder, the decoder canrealize high-speed operation that quickly responds to a skip operationby the user.

BRIEF DESCRIPTION OF THE DRAWINGS

These and the other objects, advantages and features of the inventionwill become apparent from the following description thereof taken inconjunction with the accompanying drawings which illustrate a specificembodiment of the invention. In the drawings:

FIGS. 1A, 1B and 1C show principles of stereoscopic viewing usingparallax video;

FIG. 2 shows a structure of a BD-ROM;

FIG. 3 shows an example of a structure of an AV clip stored in a file(XXX. M2TS);

FIG. 4 shows a relationship between the AV clip and the PL;

FIG. 5 shows an example of management information of an AV clip storedin a clip information file;

FIG. 6 shows a relationship between PTSs allocated to a plurality ofpictures composing a video stream for a left eye (left-eye video stream)and PTSs allocated to a plurality of pictures composing a video streamfor a right eye (right-eye video stream);

FIG. 7 shows a reference relationship between pictures;

FIG. 8 schematically shows an example of multiplexing of the left-eyevideo stream and the right-eye video stream;

FIG. 9 schematically shows another example of multiplexing of theleft-eye video stream and the right-eye video stream;

FIG. 10 schematically shows how each of the video streams is multiplexedin the AV clip;

FIG. 11 is a block diagram showing a structure of a playback apparatus2000;

FIG. 12 is a flowchart showing video decoding processing by a videodecoder 2302;

FIG. 13 shows a structure of a home theater system;

FIG. 14 is a block diagram showing an internal structure of a recordingmedium 40;

FIGS. 15A, 15B and 15C show how to effectively generate an elementarystream of 3D video;

FIG. 16 shows a structure of an index table including 3D flags;

FIG. 17 shows an example of a structure of an AV clip stored in a streamdirectory;

FIG. 18 shows a structure of a play list when the left-eye video streamand the right-eye video stream are recorded as separate digital streams;

FIGS. 19A and 19B show a difference between a focal point of the eyeswhen the user actually looks at an object and a focal point of the eyeswhen the user performs stereoscopic viewing;

FIG. 20 shows play list information when a plurality of sub paths exist;

FIG. 21 shows how the user views an object displayed on a display at thetime of playing back 2D graphics;

FIG. 22 shows how the object appears to pop out towards the user fromthe display at the time of playing back 3D graphics;

FIG. 23 shows a table in which each sub clip is in correspondence withan audio data piece, a subtitle and a menu; and

FIG. 24 shows a flowchart showing inquiry processing that inquires tothe display whether 3D graphics can be displayed.

DESCRIPTION OF NUMERAL REFERENCES 1000 BD-ROM 2000 playback apparatus2100 BD-ROM drive 2200 track buffer 2300 system target decoder 2301demultiplexer 2302 video decoder 2303 left-eye video plane 2304right-eye video plane 2305 sub video decoder 2306 sub video plane 2307PG decoder 2308 PG plane 2309 IG decoder 2310 IG plane 2311 imageprocessor 2312 image plane 2313 audio decoder 2400 plane adder 2500program memory 2600 management information memory 2700 program executionunit 2800 playback control unit 2900 user event processing unit 3000display 4000 stereoscopic glasses 5000 remote control 40 recordingapparatus 41 video encoder 42 material generating unit 43 scenariogenerating unit 44 BD program generating unit 45 multiplexing processingunit 46 format processing unit

DESCRIPTION OF PREFERRED EMBODIMENT

The following describes the embodiments of the present invention withreference to the drawings.

First Embodiment 1. Principles of Stereoscopic Viewing

Firstly, the following describes the principles of providingstereoscopic viewing using a display for home use. There are two majormethods of realizing stereoscopic viewing: a method using holographytechnology and a method using parallax video (composed of two videostreams that are based on binocular parallax).

The method using the holographic technology can allow us to view anobject in an image stereoscopically in exactly the same manner as weusually recognize a physical object. However, although technical theoryhas been established, it is very difficult, in the case of movingimages, to realize, with current technology, the stereoscopic viewingusing the holographic technology since the following computer anddisplay device are necessary: the computer capable of performing anenormous amount of calculation to generate moving images for theholography in real time, and the display device having a resolutiongreat enough to draw thousands of lines in a space as small as 1 mm.Therefore, it is a reality that the holographic technology has rarelybeen realized for commercial purposes.

In the method using parallax video, on the other hand, the stereoscopicviewing can be realized by preparing pictures for a right eye (right-eyepictures) and pictures for a left eye (left-eye pictures), and allowingthe right eye pictures to be seen only by the right eye, and theleft-eye pictures to be seen only by the left eye. FIGS. 1A, 1B and 1Cshow principles of stereoscopic viewing using parallax video. FIG. 1A isa top view showing how the user looks at a comparatively small cube infront of the user, FIG. 1B shows how the cube looks when the user looksat the cube with the left eye, and FIG. 1C shows how the cube looks whenthe user looks at the cube with the right eye. As shown in FIGS. 1B and1C, angles from which a picture is captured are different between theleft and right eyes. The stereoscopic viewing is realized by combining,in the brain, such pictures captured by the left and right eyes fromdifferent angles.

This method has a merit that stereoscopic viewing can be realized bypreparing only two pictures (one for the right eye, and the other forthe left eye) viewed from different observing points. Some technologiesusing this method have been put to practical use by technology focusingon how to allow pictures each corresponding to the right eye and theleft eye to be seen only by the corresponding eyes.

One of the technologies is called a successive separation method. Insuch method, the left-eye pictures and the right-eye pictures aredisplayed alternately on the display. When the user observes thedisplayed pictures through successive-type stereo glasses (with liquidcrystal shutters), left and right scenes are superimposed onto eachother by spectrum reaction of eyes. Thus, the user recognizes thepictures as stereoscopic pictures. More specifically, at the moment whena left-eye picture is displayed on the display, the successive typestereo glasses bring a liquid crystal shutter for the left eye into alight-transmitting state, and a liquid crystal shutter for the right eyeinto a light-blocking state. At the moment when a right-eye picture isdisplayed on the display, on the other hand, the successive-type stereoglasses bring the liquid crystal shutter for the right eye into thelight-transmitting state, and the liquid crystal shutter for the lefteye into the light-blocking state. With such method, the left-eye andright-eye pictures can be seen by corresponding eyes.

Thus, in order to alternately display left-eye pictures and right-eyepictures in the time axis direction, for example, it is necessary todisplay 48 pictures (a sum of right and left pictures) in a second inthe successive separation method while only 24 pictures are displayed ina second in a standard two dimensional movie. Accordingly, such methodis suitable for use in a display capable of performing rewriting of ascreen comparatively fast.

Another technology uses a lenticular lens. The left-eye pictures and theright-eye pictures are alternatively displayed in the time axisdirection in the above-mentioned successive separation method. However,in the technology using a lenticular lens, left-eye pictures andright-eye pictures are arranged alternatively in a longitudinaldirection on a screen simultaneously, and the lenticular lens isattached to the surface of the display. When the user views picturesdisplayed on the display through the lenticular lens, the pixelscomposing the left-eye pictures are viewed only by the left eye, and thepixels composing the right-eye pictures are viewed only by the righteye. Since it is possible to allow the right eye and the left eye toview the pictures having parallax, the user can recognize the picturesdisplayed on the display as stereoscopic pictures. Note that this is notlimited to a lenticular lens, and a device having the same function(e.g. liquid crystal element) may be used. A further alternative is amethod by which vertical polarized filters are provided on pixels forthe left eye and horizontal polarized filters are provided on pixels forthe right eye, and the user uses polarized glasses in which (i) avertical polarized filter is provided on a lens for the left eye and(ii) a horizontal filter is provided with a lens for the right eye. Thisallows the user to recognize the displayed pictures stereoscopically.

The above-mentioned method of providing stereoscopic viewing using theparallax video has been commonly used in equipment in amusement parks,and has been technically established. Therefore, such method is theclosest to being realized for household use. Other than such a method,various technologies such as a two-color separation method and the likehave been suggested as a method for providing stereoscopic viewing usingparallax video.

Although a description is given taking the successive separation methodand the polarized glasses method as examples, methods for providingstereoscopic viewing using parallax video are not limited to these, andcan be any methods that use parallax video.

2. Data Structure for Storing Parallax Video (Hereinafter Also Referredto as “3D Pictures”) for Allowing the User to Perform the StereoscopicViewing.

The following describes a data structure for storing 3D pictures in aBD-ROM which is a recording medium pertaining to the presentapplication.

FIG. 2 shows a structure of the BD-ROM. The BD-ROM is shown in a fourthtier from the top in the present figure, and a track on the BD-ROM isshown in a third tier. Although the track is usually formed in a spiralmanner from an inner circumference to an outer circumference, the trackis drawn in a laterally-expanded manner in the present figure. Thistrack consists of a read-in area, a volume area and a read-out area. Thevolume area in the present figure has a layer model having a physicallayer, a file system layer and an application layer. A top tier of FIG.2 shows an application layer format (application format) of the BD-ROMexpressed using a directory structure. As shown in the present figure,in the BD-ROM, a BDMV directory is immediately below a ROOT directory;and an index file (index. bdmv), a PLAYLIST directory, a CLIPINFOdirectory, a STREAM directory and a PROGRAM directory exist below theBDMV directory.

An index table showing a title structure is stored in a index file(index. bdmv). Titles are units of playback. For example, a main film isrecorded in a first title, a director's cut is recorded in a secondtitle, and bonus content is recorded in a third title. The user canspecify a title to play back (e.g. specifying “play back Nth title”)using a remote control or the like that comes with the playbackapparatus.

In a STREAM directory is stored a file (XXX. M2TS) including therein anAV clip in which AV contents such as video and audio are multiplexed.FIG. 3 shows an example of a structure of an AV clip stored in the file(XXX. M2TS). As shown in FIG. 3, in the file (XXX. M2TS) is stored adigital stream in a form of MPEG-2 transport stream. In such digitalstream, a video stream for the left eye (left-eye video stream), a videostream for the right eye (right-eye video stream) and the like aremultiplexed. Here, the left-eye video stream is played back as 2D video,and also is played back as video for the left eye in the case ofallowing the user to perform stereoscopic viewing of moving pictures (inthe case of playing back the digital stream as 3D video). The right-eyevideo stream is played back together with the left-eye video stream inthe case of allowing the user to perform stereoscopic viewing of movingpictures. Also, as shown in FIG. 3, 0x1011 is allocated to the left-eyevideo stream as a PID, and 0x1012 which is different from the PID of theleft-eye video stream is allocated to the right-eye video stream. Thisallows the video streams to be distinguished from each other. Each ofthe video streams is recorded after having been compressed and encodedaccording to the MPEG-2 method, MPEG-4 AVC method, SMPTE VC-1 method orthe like. An audio stream is recorded after having been compressed andencoded according to the Dolby AC-3 method, Dolby Digital Plus method,MLP method, DTS method, DTS-HD method, linear PCM method or the like.

In a PLAYLIST directory is stored a play list file (YYY. MPLS) includingtherein play list information in which a logical playback path (PL) inthe AV clip is defined. FIG. 4 shows a relationship between the AV clipand the PL. As shown in FIG. 4, play list information is composed of oneor more play item (PI) information pieces. Each of the play iteminformation pieces represents a playback section in the AV clip. Each ofthe play item information pieces is identified by a play item ID, and iswritten in playback order in the play list.

Also, the playlist information pieces include entry marks each showing aplayback start point. The entry marks can be provided in the playbacksections defined in the play item information pieces. Also, as shown inFIG. 4, each of the entry marks is located in a position to be aplayback start point in the play item information, and is used for cueplayback. For example, chapter playback can be performed by providing anentry mark to a position to be a start of each chapter in a movie title.Note that a playback path of a set of play item information pieces isdefined as a main path.

In a CLIPINFO directory is stored a clip information file (XXX. CLPI)including therein management information on the AV clip. FIG. 5 shows anexample of management information on the AV clip stored in the clipinformation file. As shown in FIG. 5, the management information on theAV clip is in one to one correspondence with the AV clip, and iscomposed of clip information, stream attribute information and entrymaps.

In each of the entry maps are written entry map header information,table information, and another table information relating to a videostream (sub video). Here, the table information shows pairs each ofwhich is composed of (i) a PTS (Presentation Time-Stamp) showing adisplay time of a head of each of GOPs that compose the left-eye videostream, and (ii) a SPN (Source Packet Number) showing a start positionof each of the GOPs in the AV clip. Here, information on each of thepairs of PTSs and SPNs shown in one row is called an entry point. Also,values starting from 0 each of which is incremented by one in order arecalled an entry point ID (hereinafter also referred to as “EP_ID”).

Also, in the entry map header information are stored information on aPID of the left-eye video stream, the number of entry points and thelike.

By referring to such entry maps, the playback apparatus can, when theplayback start point is specified by time, convert time information intoaddress information, and specify a packet point on an AV clip. Thepacket point corresponds to an arbitrary point on a time axis of thevideo stream.

In a PROGRAM directory is stored a BD program file (AAA. PROG) includingtherein a program for defining a dynamical scenario.

In a BD-ROM, although a proprietary interpreter-type program called“command navigation” is used, a language system is not an essence of thepresent invention. Therefore, a program written by a general-purposeprogramming language such as Java or Java script is also possible. Aplay list to be played back is specified by such program.

3. PTSs of a Left-Eye Video Stream and a Right-Eye Video Stream

The following describes PTSs of the left-eye video stream and theright-eye video stream for allowing the user to perform stereoscopicviewing of the moving pictures.

FIG. 6 shows a relationship between a display time (PTS) allocated toeach of a plurality of pictures that compose the left-eye video stream,and a display time (PTS) allocated to each of a plurality of picturesthat compose the right-eye video stream. The pictures composing theleft-eye video stream (left-eye pictures) are in one to onecorrespondence with the pictures composing the right-eye video stream(right-eye pictures) (e.g. a picture L1 and a picture R1 as shown inFIG. 6 correspond to each other). PTSs are set such that the left-eyepictures that are in correspondence with the right-eye pictures aredisplayed before the corresponding right-eye pictures. Also, PTSs forthe left-eye pictures and PTSs for the right-eye pictures are set so asto alternate each other on a time axis. This can be realized by settingthe PTSs such that the left-eye pictures and the right-eye pictures thatare in a reference relationship of an intra-picture prediction codingare displayed alternately.

The following describes the left-eye pictures and the right-eye picturesthat are in the reference relationship of the intra-picture predictioncoding. The right eye pictures are compressed by an intra-pictureprediction coding that is based on redundancy between views in additionto an intra-picture prediction coding that is based on redundancy in atime-axis direction. That is, the right-eye video pictures arecompressed with reference to the corresponding left-eye pictures. Thefollowing describes the reasons for this. The left-eye pictures and theright-eye pictures strongly correlate to each other since objects of theleft-eye pictures and the right-eye pictures are the same though viewsthereof are different. Therefore, the data amount of right-eye videostream can be greatly reduced compared to the data amount of theleft-eye video stream by performing the intra-picture prediction codingbetween the views.

FIG. 7 shows a reference relationship between pictures. As shown in FIG.7, a P₀ picture of the right-eye video stream refers to an I₀ picture ofthe left-eye video stream. A B₁ picture of the right-eye video streamrefers to a Br₁ picture of the left-eye video stream. A B₂ picture ofthe right-eye video stream refers to a Br₂ picture of the left-eye videostream. A P₃ picture of the right-eye video stream refers to P₃ pictureof the left-eye video stream.

Since the left-eye video stream does not refer to the right-eye videostream, the left-eye video stream alone can be played back (i.e. theleft-eye video stream can be played back as 2D video). However, theright-eye video stream alone cannot be played back since the right-eyevideo stream refers to the left-eye video stream.

The following describes the time interval between PTSs of the left-eyepictures and PTSs of the corresponding right-eye pictures using FIG. 6.When 3D graphics is played back using the successive separation method,each PTS for the right-eye picture needs to be set at an interval thatsatisfies the following equation with respect to a left-eye pictureshown by a certain time (PTS):

PTS (for the right eye)=PTS (for the left eye)+1/(the number of framesper second×2).

When a frame rate is 24 p, for example, it is indicated that 24 picturesare displayed in a second. Therefore, the interval (display delay)between the left-eye pictures and the corresponding right-eye picturesis 1/48 seconds.

Thus, the right-eye pictures need to be synchronized with thecorresponding left-eye pictures with a display delay ( 1/48 seconds).

Therefore, in the case of multiplexing the left-eye video stream and aright-eye video stream into a transport stream, multiplexing may beperformed such that the left-eye pictures are arranged in a vicinity ofthe corresponding right-eye pictures based on PTSs and DTSs. Here, thePTSs show display times of pictures in each GOP of each of the videostreams, and the DTSs show decoding times of pictures in each GOPs ofeach of the video streams.

When the streams are multiplexed in such way, required left-eye andright-eye pictures can be obtained at a necessary time if a transportstream is read in order from the head.

However, in some cases, playback is performed from a time point otherthan a time point of the head of the stream because of a skip operationor a time-specifying jumping operation. In such cases, since entrypoints are written in each of the entry maps in units of GOPs, GOPboundaries between the left-eye pictures and the right-eye pictures needbe considered when performing multiplexing.

4. Multiplexing a Left-Eye Video Stream and a Right-Eye Video Stream

The following describes how to perform multiplexing of the left-eyevideo stream and the right-eye video stream with random access to an AVclip taken into consideration.

Firstly, setting needs to be made such that the GOPs of the left-eyevideo stream and the GOPs of the right-eye video stream have the sametemporal interval. Also, the GOPs of the left-eye video stream need tobe in one to one correspondence with the GOPs of the right-eye videostream. In such way, the left-eye video stream can be synchronized withthe right-eye video stream in units of GOPs.

Also, since an entry map (PID=0x1011) is set for the left-eye videostream as shown in FIG. 5, each time information piece (PTS) in theentry map shows a playback start time of a head of each GOPs of theleft-eye video stream, and each address information piece (SPN) shows anaddress of a head of packets of each GOP of the left-eye video stream inthe AV clip. Since the playback apparatus reads data from a positionshown by such address in the AV clip, arrangement needs to be made suchthat each of the GOPs of the right-eye video stream succeeds the head ofthe packets that compose each of the corresponding GOPs of left-eyevideo stream.

FIG. 8 schematically shows an example of multiplexing of the left-eyevideo stream and the right-eye video stream. As shown in FIG. 8, theleft-eye video stream and the right-eye video stream are multiplexed inunits of the GOPs in a form that GOPs of the left-eye video streamprecede the GOPs of the corresponding right-eye video stream.

In such way, when playback starts, for example, from LGOP2 of theleft-eye video stream in FIG. 8, RGOP2 of the right-eye streamcorresponding to LGOP2 of the left-eye video stream can be read withouta problem if the playback starts from a head of packets that composeLGOP2. Therefore, the left-eye video stream and the right-eye videostream can be played back as 3D graphics at such playback start time.

As shown in the above, by adding, when multiplexing is performed,restrictions that (i) GOPs of the left-eye video stream and GOPs of theright-eye video stream have the same temporal interval, and (ii) a headof each GOP of the left-eye video stream always precedes a head of eachcorresponding GOP of the right-eye video stream, it is ensured that theleft-eye video stream and the right-eye video stream can be played backas 3D graphics no matter which time point shown by the entry map theplayback starts from.

The following describes another example of multiplexing the left-eyevideo stream and the right-eye vide stream. FIG. 9 schematically showsanother example of multiplexing the left-eye video stream and theright-eye vide stream. FIG. 8 and FIG. 9 are common in that a head ofeach GOP of the left-eye video stream always precedes a head of eachcorresponding GOP of the right-eye video stream when multiplexing isperformed. However, while there is no restriction on an arrangement ofan end position of each GOP for the left eye (left-eye GOP) and anarrangement of an end position of each corresponding GOP for the righteye (right-eye GOP) in FIG. 8, FIG. 9 is different from FIG. 8 in thatthe right-eye GOP is located between the header packet and the endpacket of the corresponding left-eye GOP succeeds in FIG. 9. In suchway, even when an AV clip is cut according to GOP boundaries of theleft-eye video stream, the AV clip can be cut without cutting GOPs ofthe right-eye video stream.

FIG. 10 schematically shows how each video stream is multiplexed in theAV clip when a right-eye GOP is located between a header packet and anend packet of the corresponding left-eye GOP.

Firstly, the left-eye video stream composed of a plurality of videoframes is converted into a PES packet string, and the PES packet stringis converted into a TS packet string. Similarly, the right-eye videostream composed of a plurality of video frames is converted into a PESpacket string, and the PES packet string is converted into a TS packetstring.

As shown in FIG. 10, a header packet (L11) of GOPs (LGOP1) of theleft-eye video stream is arranged first when the left-eye video streamand the right-eye video stream are multiplexed. Packets (R11, R12 andR13) of a GOP (RGOP1) of the right-eye video stream corresponding toLGOP1 succeed an arrangement position of the packet (L11) of LGOP1, andprecede an arrangement position of an end packet (L16) of LGOP1. Aheader packet (L21) of the next GOP (LGOP2) of the left-eye video streamfollows the end packet (L16) of LGOP1. As with the above, packets (R21,R22 and R23) of a GOP (RGOP2) of the right-eye video streamcorresponding to LGOP2 precede an end packet (L26) of LGOP2. Bymultiplexing the left-eye video stream and the right-eye video stream insuch way, a digital stream is generated that ensures that GOPs of theright-eye video stream are not cut when the AV clip is cut according tothe GOP boundaries of the left-eye video stream.

5. Playback Apparatus

The following describes a playback apparatus that plays back a BD-ROM1000 storing 3D graphics. FIG. 11 is a block diagram showing a structureof a playback apparatus 2000. As shown in FIG. 11, the playbackapparatus 2000 is composed of a BD-drive 2100, a track buffer 2200, asystem target decoder 2300, a plane adder 2400, a program memory 2500, amanagement information memory 2600, a program execution unit 2700, aplayback control unit 2800, and a user event processing unit 2900.

The BD-ROM drive 2100 reads data from the BD-ROM 1000 based on a readrequest inputted from the playback control unit 2800. The AV clip readfrom the BD-ROM 1000, management information (an index file, a play listfile and a clip information file), a BD program file are transferred tothe track buffer 2200, the management information memory 2600 and theprogram memory 2500, respectively.

The track buffer 2200 is a buffer composed of a memory and the likestoring therein AV data clip inputted from the BD-ROM drive 2100.

The system target decoder 2300 performs multiple separation processingon the AV clip stored in the track buffer 2200, and performs decodingprocessing on streams. The playback control unit 2800 transfers, to thesystem target decoder 2300, information (a codec type, a streamattribute and the like) included in the AV clip that are necessary fordecoding the streams.

The system target decoder 2300 is specifically composed of ademultiplexer 2301, a video decoder 2302, a left-eye video plane 2303, aright-eye video plane 2304, a sub video decoder 2305, a sub video plane2306, a presentation graphics decoder (PG decoder) 2307, a presentationgraphics plane (PG plane) 2308, an interactive graphics plane decoder(IG decoder) 2309, an interactive graphics plane (IG plane) 2310, animage processor 2311, an image plane 2312 and an audio decoder 2313.

The demultiplexer 2301 extracts TS packets stored in the track buffer2200, and obtains a PES packet from the extracted TS packets. Thedemultiplexer 2301 outputs the PES packet to one of the video decoder2302, the sub video decoder 2305, the PG decoder 2307, the IG decoder2309 and the audio decoder 2313, based on PIDs (packet identifiers)included in TS packets. Specifically, the PES packet obtained from theextracted TS packets is transferred to: the video decoder 2302 if thePIDs included in the TS packets are 0x1011 or 0x1012, the sub videodecoder 2305 if the PIDs are 0x1B00, the audio decoder 2313 if the PIDsare 0x1100 or 0x1101, the PG decoder 2307 if the PIDs are 0x1200 or0x1201, and to the IG decoder 2309 if the PIDs are 0x1400.

The video decoder 2302 decodes the PES packet inputted from thedemultiplexer 2301 to obtain uncompressed pictures, and writes thepictures in one of the left-eye video plane 2303 and the right-eye videoplane 2304. The following describes an operation of the video decoder2302. FIG. 12 shows a flowchart showing video decoding processing by thevideo decoder 2302. Receiving the PES packet from the demultiplexer 2301(step S101), the video decoder 2302 decodes the received PES packet toobtain uncompressed pictures (step S102).

The video decoder 2302 judges whether the uncompressed pictures composea left-eye video frame or a right-eye video frame (step S103). Suchjudgment is made as follows. When, for example, the demultiplexer 2301transmits the PES packet to the video decoder 2302, a flag showingwhether the PES packet is for the left-eye video stream or the right-eyevideo stream, based on the PIDs included in the TS packets is added; andthe video decoder 2302 judges whether the flag shows the PES packet isfor the left-eye video stream.

When judging that the uncompressed pictures compose the left-eye videoframe (step S103: Yes), the video decoder 2302 writes the pictures inthe left-eye video plane 2303 (step S104).

When judging that the uncompressed pictures compose the right-eye videoframe (step S103: No), the video decoder 2302 writes the pictures in theright-eye video plane 2304 (step S105).

In FIG. 11, the left-eye video plane 2303 is a plane for storinguncompressed pictures for the left eye. The plane is a memory area forstoring, in the playback apparatus, pixel data corresponding to onescreen. Resolution in the video plane is 1920×1080, and picture datastored in such video plane is composed of pixel data expressed by a YUVvalue of 16 bits.

The right-eye video plane 2304 is a plane for storing uncompressedpictures for the right eye.

The video decoder 2302 writes pictures in the left-eye video plane 2303and the right-eye video plane 2304 at a time shown by a PTS of the videoframe.

The sub video decoder 2305 has the same structure as the video decoder2302, decodes the video frame inputted from the demultiplexer 2301, andwrites the uncompressed pictures in the sub video plane 2306 at a timeshown by the display time (PTS).

The sub video plane 2306 is a plane for storing uncompressed pictures ofsub video.

The PG decoder 2307 decodes a presentation graphics stream inputted fromthe demultiplexer 2301, and writes uncompressed graphics data in the PGplane 2308 at the display time (PTS).

The PG plane 2308 is a plane for storing graphics data.

The IG decoder 2309 decodes an interactive graphics stream inputted fromthe demultiplexer 2301, writes the uncompressed graphics data in the IGplane 2310 at a time shown by the display time (PTS).

The IG plane 2310 is a plane for storing graphics data.

The image processor 2311 decodes graphics data (PNG.JPEG) inputted fromthe program execution unit 2700, and outputs the decoded graphics datato the image plane 2312. Decoding timing of the image plane 2312 isindicated by the program execution unit 2700 when the graphics data isdata for menu. The decoding timing of the image plane 2312 is indicatedby the playback control unit 2800 when the graphics data is data forsubtitles.

The image plane 2312 is a plane for storing graphics data (PNG.JPEG).

The audio decoder 2313 decodes the PES packets inputted from thedemultiplexer 2301, and outputs uncompressed audio data.

The plane adder 2400 (i) determines in which of the left-eye video plane2303 and the right-eye video plane 2304 pictures are written at a timeshown by a PTS, and (ii) generates a video signal by superimposing theselected plane, the sub video plane 2306, the PG plane 2308, the IGplane 2310 and the image plane 2312 instantly, and outputs the videosignal to a display of TV and the like. The video signal includes a flagshowing which of the left-eye picture or a right-eye picture issuperimposed.

The program memory 2500 is a memory for storing a BD program fileinputted from the BD-ROM drive 2100.

The management information memory 2600 is a memory for storing an indextable, management information and play list information inputted fromthe BD-ROM drive 2100.

The program execution unit 2700 executes a program stored in the BDprogram file stored in the program memory 2500. Specifically, theprogram execution unit 2700 (i) instructs the playback control unit 2800to play back a play list based on the user event inputted from the userevent processing unit 2900, and (ii) transfers, to the system decoder2300, PNG.JPEG for a menu and game graphics.

The playback control unit 2800 has functions of controlling playback ofthe AV clip by controlling the BD-ROM drive 2100 and the system targetdecoder 2300. For example, the playback control unit 2800 controls theplayback processing of the AV clip with reference to play listinformation stored in the management information memory 2600, based onthe playback instruction inputted from the program execution unit 2700.Also, in the case of random access, the playback control unit 2800specifies a start position of a GOP corresponding to time informationregistered in entry maps stored in the management information memory2600, and instructs the BD-ROM drive 2100 to start reading from thestart position. In such way, processing can be performed efficientlywithout analyzing the AV clip. Furthermore, the playback control unit2800 performs acquisition of information on a state and setting of thestate.

In accordance with a key operation performed on a remote control or afront panel of a playback apparatus, the user event processing unit 2900outputs, to the program execution unit 2700, a user event showing suchoperation.

This concludes the structure of the playback apparatus 2000.

6. Structure of Viewing 3D Graphics at Home

The following describes a structure of viewing 3D graphics at home. Inorder for the user to view 3D graphics at home, a display capable ofdisplaying 3D graphics outputted from the playback apparatus, andglasses for stereoscopic viewing (stereoscopic glasses) are necessary inaddition to the above-stated BD-ROM and the playback apparatus 2000 thatplays back the BD-ROM. FIG. 13 shows a home system composed of theBD-ROM 1000, the playback apparatus 2000, the display 3000, stereoscopicglasses 4000, and a remote control 5000.

The playback apparatus 2000 and the display are connected to each othervia, for example, a HDMI (High-Definition Multimedia Interface) cable.

The display 3000 displays a video signal inputted from the playbackapparatus 2000 according to time division. Since the playback apparatus2000 inputs left-eye pictures and right-eye pictures alternately, thedisplay 3000 displays the left-eye pictures and the right-eye picturesalternately in a time axis direction.

The display 3000 is different from a display for displaying 2D graphicsin that the display 3000 needs to be capable of quickly switching ascreen in order to alternately display left-eye pictures and right-eyepictures compared to such display for 2D graphics. For example, whilemost movies are shot using 24p (24 frames in a second), 48 frames needto be rewritten in a second in the case of the 3D graphics since 24frames for the left-eye video and 24 frames for the right-eye video needto be alternately displayed in a second. Also, the display capable ofplaying back 3D graphics has another feature in which setting is madesuch that edge enhancement processing performed in an existing consumerTV is not performed while the 3D graphics is displayed. This is because,in viewing 3D graphics, edge positions of the left-eye pictures and theright-eye pictures are important, and stereoscopic viewing cannot beprovided properly if imbalance of the left-eye pictures and theright-eye pictures arises because the edge lines become thicker orthinner due to the edge enhancement and the like.

Also, the display 3000 transmits, to the stereoscopic glasses 4000, acontrol signal showing which of the left-eye picture and the right-eyepicture is displayed on the display, based on a flag included in a videosignal inputted via the HDMI cable.

As described in the principles of stereoscopic viewing in the firstembodiment, the stereoscopic glasses 4000 are used in viewing 3Dgraphics in the successive separation method, and are special glassesincluding liquid crystal shutters. The stereoscopic glasses 4000 switch,based on the control signal, each of the liquid crystal shutter for theleft eye and the liquid crystal shutter for the right eye between alight-transmitting state and a light-blocking state. Specifically, whenreceiving, from the display, a control signal showing that a left-eyepicture is being displayed, the stereoscopic glasses bring the liquidcrystal shutter for the left eye into the light-transmitting state, andthe crystal liquid shutter for the right eye into the light-blockingstate. When receiving a control signal showing that a right-eye pictureis being displayed, the stereoscopic glasses bring the liquid crystalshutter for the right eye into the light-transmitting state, and thecrystal liquid shutter for the left eye into the light-blocking state.

According to the above-stated embodiment, it is possible, even whenplayback starts from an arbitrary PTS on the time axis of the digitalstream, to reliably allow the user to perform stereoscopic viewing ofmoving pictures. This is because GOPs of the left-eye video stream andthe corresponding GOPs of the right-eye video stream in GOP pair regionswhose beginning are indicated by the SPNs corresponding to the PTSs inthe recording area of the digital stream on the time axis.

Second Embodiment

The present embodiment describes the recording apparatus and therecording method pertaining to the present invention.

The recording medium (i) is located in a production studio fordistributing film contents, (ii) generates (a) a digital stream whichhas been compressed and encoded according to the MPEG standards, and (b)a scenario on which how a movie title is played back is written, and(iii) generates a volume image for a BD-ROM including such data pieces.The recording medium generates a recording medium described in the firstembodiment.

FIG. 14 is a block diagram showing an internal structure of therecording medium 40. As shown in the present figure, the recordingapparatus 40 is composed of a video encoder 41, a material generatingunit 42, a scenario generating unit 43, a BD program generating unit 44,a multiplexing processing unit 45 and a format processing unit 46.

The video encoder 41 encodes images such as uncompressed bitmaps ofleft-eye pictures and images such as uncompressed bitmaps of right-eyepictures according to compressing method such as MPEG4-AVC, MPEG2 andthe like, and generates a left-eye video stream and a right-eye videostream. In that case, setting is made such that GOPs of the left-eyevideo stream and GOPs of the right-eye video stream have the sametemporal interval.

The following describes a method of efficiently generating an elementarystream of 3D video.

When compressing video for package media such as a DVD and a BD-ROM, ingeneral video compressing technology, compression rate is increased byperforming compression making use of similarities between previousand/or subsequent pictures. At this time, an enormous amount of time isneeded for encoding in order to search for similar parts in the previousand/or subsequent images. FIGS. 15A, 15B and 15C show how to efficientlygenerate an elementary stream of 3D video. FIG. 15A shows a left-eyepicture at a certain time point, and FIG. 15B shows the left-eye pictureat a time point immediately after the time point of the picture shown inFIG. 15A. The video encoder 41 searches, in the left-eye picture shownin FIG. 15B, for a cube or a circle exists in the left-eye picture shownin FIG. 15A. At this time, it is necessary to perform searching in thewhole screen shown by FIG. 15B in order to perform searching in thelargest range at the time of encoding the left-eye pictures. In the caseof a general encoding process, the right-eye pictures are encoded byperforming the same procedures as the case of encoding the left-eyepictures. That is, it is necessary to perform searching in the wholescreen as shown in FIG. 15B.

As evident from the above, it takes twice as much compressing time inencoding 2D video as in encoding 3D video (a string of left-eye videoframes and a string of right-eye video frames) since it is necessary tocompress each string of video frames separately.

The video encoder 41 writes, in the table, to which direction and howfar each search target moves at the time of encoding the left-eye video.This shortens encoding time since, by referring to the table, it is notnecessary to perform searching in the whole screen in the case ofencoding the right-eye pictures which are very similar to the left-eyepictures. FIG. 15C shows a table showing to which direction and how fareach search target moves. In the case of encoding the right-eyepictures, a search range can be narrowed down by referring to the tableshown in FIG. 15C.

Note that although a description is given using figures such as the cubeand the circle in order to make the description simple, a movingdirection and a distance may be recorded, at the time of actualencoding, for each of rectangle and square areas (8×8, 16×16 and thelike) as with the case of the geometric figures.

Back in FIG. 14, the material generating unit 42 generates streams suchas an audio stream, a presentation graphics stream, an interactivegraphics stream and the like. More specifically, the material generatingunit 42 generates an audio stream by encoding uncompressed Linear PCMaudio or the like according to a compressing method such as the AC3method and the like.

Also, the material generating unit 42 generates a presentation graphicsstream which is a format of a subtitle stream complying with the BD-ROMstandard, based on a subtitle information file including subtitlingeffects such as a subtitle image, display timing, fade-in/fade-out andthe like.

Furthermore, the material generating unit 42 generates an interactivegraphics stream which is a format of a menu screen complying with theBD-ROM standard, based on bitmap images used for menus, and a menu fileon which transition and the display effect of buttons arranged on themenu are written.

The scenario generating unit 43 generates a scenario, using a formatcomplying with the BD-ROM standard, according to information on eachstream generated in the material generating unit 42, and the useroperation. The scenario is a file such as an index file, a movie objectfile, a play list file or the like.

Also, the scenario generating unit 43 generates a parameter file forrealizing multiplexing processing. In the parameter file, which streameach AV clip is composed of is written. Each file such as an index file,a movie object file, a play list file and the like to be generated hasthe same data structure the structure described in the first embodiment.

The BD program generating unit 44 generates a program of a BD program.The BD program generating unit 44 generates (i) source codes of the BDprogram, according to the request from a user, via a user interface suchas GUI or the like, and (ii) a BD program file.

The multiplexing processing unit 45 multiplexes a plurality of streamssuch as a left-eye video stream, aright-eye video stream, an audiostream, a presentation graphics stream, an interactive graphics streamand the like that are written on the BD-ROM scenario data to generate anAV clip according to the MPEG2-TS method. In that case, a left-eye videostream and a right-eye video stream are multiplexed in units of GOPs ina manner that a head of GOPs of the left-eye video stream precedes acorresponding GOP of the right-eye video stream.

Also, in the case of creating an AV clip, a clip information file whichmakes a pair with the AV clip is generated simultaneously. Generation ofthe clip information file by the multiplexing processing unit 45 isperformed by the following method. The multiplexing processing unit 45generates entry maps in each of which a storing position of a headerpacket of each GOP of the left-eye video stream is in correspondencewith a display time of the header packet of each of the GOPs, andgenerates a clip information file by pairing, in one to onecorrespondence, the generated entry maps with attribute informationshowing audio attributes, video attributes and the like of each streamincluded in the AV clip. The structure of the clip information file hasthe same data structure as the structure described in the firstembodiment.

The format processing unit 46 (i) arranges: according to the formatcomplying with the BD-ROM standard, the BD-ROM scenario data generatedin the scenario generating unit 43; the BD program file generated in theBD program generating unit 44; and the AV clip and the clip informationfile generated in the multiplexing processing unit 45, and (ii)generates a disc image according to the UDF format which is a filesystem complying with the BD-ROM standard. The format processing unit 46converts the generated disc image into data for BD-ROM pressing, andperforms a pressing step on this data, thereby manufacturing a BD-ROM.

Third Embodiment

The present embodiment describes a case in which 2D pictures and 3Dpictures exist together on the BD disc. If all the pictures of moviecontent recorded on the BD-ROM disc are either 3D or 2D, the user canchoose either to wear or not to wear the above-mentioned glasses forstereovision according to discs. However, when 2D pictures and 3Dpictures exist together on one disc, the user needs to wear or take offthe glasses with certain timing.

From the perspective of the user, it is very difficult to see when towear and take off the glasses if, without notification, 2D picturessuddenly switch to 3D pictures, or the 3D pictures suddenly switch tothe 2D pictures, on the other hand.

In order to solve this, a 3D flag showing whether a title is 2D video or3D video is provided with each title. When a title changes, it is judgedwhether the changed title is 2D video or 3D video based on the 3D flags.Then the user is notified of the result of the judgment. FIG. 16 shows astructure of an index table including 3D flags. As shown in FIG. 16, a3D flag showing whether video in each title is 2D or 3D is provided witheach title. Switching between titles is performed by a user operationusing a remote control, a command or the like. The playback apparatuscan notify the user of timing of wearing or taking off the stereoscopicglasses by an OSD (On Screen Display), an audio assist or the like byreferring to the above-mentioned 3D flags corresponding to therespective titles at the time of transition of the titles. This meansthat 2D video and 3D video do not exist together in one title. From theperspective of those creating commercial discs, there is a merit that itis possible to have the BD-ROM player prompt the user to wear or takeoff the glasses by clearly dividing pictures into 2D pictures and 3Dpictures for each title.

Note that although a description is given, in the above, of the casewhere a flag showing whether pictures are 2D or 3D is provided with eachcorresponding title, a 3D flag may be provided with each playlist, eachplay item or each AV stream.

Supplementary Explanations

Although a description is given of a recording medium pertaining to thepresent invention in the above based on the embodiments, it is needlessto say that the present invention is not limited to the above-statedembodiments.

(Modification 1)

In the first embodiment, the left-eye video stream and the right-eyevideo stream are multiplexed into one digital stream. However, themodification 1 describes a case in which the left-eye video stream andthe right-eye video stream are not multiplexed, and are recorded asseparate digital streams in the recording medium.

Firstly, the following describes an AV clip when the left-eye videostream and the right-eye video stream are recorded as separate digitalstreams in the recording medium. FIG. 17 shows an example of structuresof an AV clip stored in a STREAM directory.

An AV clip for the left eye (left-eye AV clip) is the same as the AVclip shown in FIG. 3, except that the left-eye AV clip does not includethe right-eye video stream in the present modification. The left-eyevideo stream is stored in the left-eye AV clip. The left-eye videostream (main video) is played back: as 2D video when played back in aplayback apparatus that plays 2D video; and as 3D video when played backin a playback apparatus capable of playing back 3D video.

The right-eye video stream is stored in an AV clip for the right eye(sub clip). The right-eye video stream is played back, as right-eyevideo, together with the left-eye video stream when 3D video is playedback in the playback apparatus capable of playing back the 3D video.

The following describes a structure of a play list when the left-eyevideo stream and the right-eye video stream are recorded as separatedigital streams in the recording medium. The play list information hasone or more sub play item (Sub PI) information pieces in addition to amain path which is a playback path of a series of play item informationpieces. A playback path of the series of sub play items which is playedback in synchronization with the main path is defined as a sub path.Each of the sub play item information pieces shows a playback section ofa sub clip. A playback section of each of the sub play item informationpieces is shown on the same time axis as the main path.

FIG. 18 shows a structure of the play list when a left-eye video streamand a right-eye video stream are recorded as separate digital streams.2D play list information is play list information when the left-eyevideo stream is played back as 2D video, and 3D play list information isplay list information when the left-eye video stream and the right-eyevideo stream are played back as 3D video. As shown in FIG. 18, a mainpath of each of the 2D play list information and the 3D play listinformation refers to the AV clip storing therein the left-eye videostream. The 3D play list information has a sub path in addition to themain path. The sub path refers to a sub clip storing therein theright-eye video stream, and is set so as to be synchronized with themain path on the time axis. With such structure, the 2D play listinformation and the 3D play list information can share the AV clipstoring therein the left-eye video stream. Also, in the 3D play listinformation, the left-eye video and the right-eye video can besynchronized with each other on the time axis.

The following describes a clip information file when the left-eye videostream and the right-eye video stream are recorded as separate digitalstreams.

Since the left-eye video stream and the right-eye video stream areseparate digital streams, a clip information file exists for each of thedigital video streams. Basically, both of the clip information fileshave the same structure as the clip information file described in thefirst embodiment. Therefore, an entry map is set for the right-eye videostream, too. In the entry map for the right eye (right-eye entry map)are written entry map header information and table information showingpairs each of which is composed of (i) a PTS showing a display time of ahead of each of GOPs that compose the right-eye video stream and (ii) aSPN showing a start position of each of the GOPs in the sub clip. Ineach entry point in the table information is registered a picture of theright-eye video stream in the sub clip. The picture of the right-eyevideo stream in the sub clip has a PTS which is a value obtained byadding, to a display delay of ( 1/48) seconds, a PTS specified by eachentry point of the left-eye video stream in the AV clip. This enablesthe playback apparatus, when a certain time is specified, to obtainstart addresses of the GOPs in the right-eye video stream and theleft-eye video stream, corresponding to the specified time.

The following describes the physical file arrangement on the BD-ROM. TheAV clip storing therein the left-eye video stream and the sub clipstoring therein the right-eye video stream are divided into extents(e.g. GOP units), and arranged in an interleave manner. GOPs of theleft-eye video stream and GOPs of the right-eye video stream have thesame temporal interval. Also, a preceding flag is set for the entry mapheader information of the entry map, for example. Here, the precedingflag shows which of a GOP of the left-eye video stream and thecorresponding GOP of the right-eye video stream precedes. This enablesthe playback apparatus to refer to the preceding flag to indicate, tothe BD-ROM, which GOP of the left-eye video stream and the correspondingGOP of the right-eye video stream should be read first. That is, it ispossible to start reading from the GOP of the video stream indicated bythe preceding flag.

Therefore, even if the left-eye video stream and the right-eye videostream are not multiplexed, and are recorded as separate digitalstreams, the GOP of the left-eye video stream and the corresponding GOPof the right-eye video stream are located at the playback start positiononwards indicated by the playback apparatus. Therefore, it is possibleto reliably allow the user to perform the stereoscopic viewing of themoving pictures.

(Modification 2)

Although the 3D play list information has one sub path in addition tothe main path in the modification 1, a modification 2 describes a casein which the 3D play list information includes a plurality of sub paths.

Firstly, the following describes what makes the user feel uncomfortablewhen performing stereoscopic viewing with use of parallax video. FIGS.19A and 19B show a difference between focal points of eyes when actuallylooking at an object and focal points of eyes when performingstereoscopic viewing. FIG. 19B shows how the user actually looks at theobject. As shown in FIG. 19B, a left eye 15 and a right eye 16 focus ona position of an object 17. That is, for the user observing the object17, a position in which the user focuses his/her eyes toward and aposition in an empty space in which the user recognizes the object arethe same.

On the other hand, FIG. 19A shows how the user performs stereoscopicviewing by parallax video. While a left eye 11 and a right eye 12 focuson a display 14, the image 13 viewed stereoscopically is recognized inthe brain in a manner that an image is formed in a point at theinterception of lines of vision from both of the eyes to the displaywhen the user looks at the display with both of the eyes. That is, whileboth of the eyes focus on the display 14, a position in which the userrecognizes the 3D object 13 is a position which is popped out from thedisplay, and such difference between a focus position and an objectrecognizing position causes a sense of discomfort and tiredness when theuser recognizes 3D graphics using parallax video.

Also, in general, it is known that the sense of discomfort and thetiredness increase as a difference between (i) a position in which eyesare actually focused (display position) and (ii) a position in which theuser recognizes an object as a 3D object in the brain becomes larger.

One way of realizing a 3D imagery using the parallax video with smallburden imposed on the user is to store, in the recording medium, aplurality of right-eye video streams each having a different pop-outdistance (an angle of an object) as separate AV clips, and to let theuser select a desirable distance. That is, two AV clips are prepared,and the user can choose an appropriate AV clip. Here, one of the AVclips is for users who are used to viewing 3D video, or users who wantto enjoy realistic sensation by observing more popped out 3D object, andanother one of the AV clips is for users who are not used to viewing 3Dgraphics, and is an AV clip in which discomfort caused when the usersview the 3D images is reduced by suppressing the pop-out distance fromthe display.

FIG. 20 shows play list information when a plurality of sub paths exist.The play list information shown in FIG. 20 refers to a sub clip set 1and a sub clip set 2. Each of the sub clip sets stores therein aright-eye video stream though the right-eye video streams havingdifferent angles of an object (pop-out distances). Each of the sub pathsis synchronized with a main path, and is provided with an ID accordingto the order of being registered in the play list information. Such IDsare used, as sub path IDs, for distinguishing between the sub paths.

A sub path having a sub path ID of “0” refers to the sub clip set 1, anda sub path having a sub path ID of “1” refers to the sub clip set 2. The3D play list information includes the plurality of sub clips havingdifferent pop-out levels, and switching is performed between sub pathsthat are played back in synchronization with a main path storing thereinthe left-eye video stream, based on a size of a display screen and auser operation. In such way, stereoscopic viewing can be performed usingparallax video which the user feels conformable with.

(Modification 3)

The modification 2 describes the case in which the 3D play listinformation includes the plurality of sub paths. The presentmodification describes a case in which audio data, a subtitle and a menuare combined with each of the sub clips.

FIG. 21 shows how a user 21 views an object 22 displayed on a display 23when 2D graphics is played back. At this time, when sound is emittedfrom the object 22, in most of the movies, realistic sensation iscreated by a combination of video and audio in order to let the userfeel as if the sound were emitted from the position of the object 22 by(i) adjusting phase of sound and sound pressure emitted from a pluralityof loudspeakers, and (ii) localizing the sound in a manner that thesound seems to be emitted from the position of the object 22.

On the other hand, FIG. 22 shows how the same object appears to pop outtowards the side of a user 31 from a display 33 when 3D video is playedback. The object is rendered (i) in a position 34 in the left-eye videostream, and (ii) in a position 35 in the right-eye video stream. In suchway, the user 31 recognizes the object as if the object were in aposition 36 in empty space, and appears to pop out. At this time, ifsound emitted from the object is localized in the position 32 using thesame sound used in the case of 2D graphics, the sound of the objectsounds to the user as if the sound of the object were emitted from theposition 32 although the user recognizes that the object is in theposition 36.

In order to solve the above-stated problem, in order for the user tohear sound from the position in which the user recognizes the object,audio data pieces are stored in one to one correspondence with aplurality of right-eye video streams in which angles from which objectsare looked at are different.

Furthermore, in addition to the audio data pieces, subtitles and menusare stored in one to one correspondence with the right-eye video streamsin the same manner. Subtitles and menus corresponding to the right-eyevideo streams having different pop-out levels (levels showing how farobjects pop out) can be displayed by storing combinations of theright-eye video streams and corresponding subtitles and menus havingdifferent pop-out levels (which do not hurt the front and backrelationship between video and subtitles or menus) according to thepop-out levels of parallax video pieces. This makes it possible toincrease the realistic sensation.

FIG. 23 shows a table in which sub clips are in one to onecorrespondence with audio data pieces, subtitles and menus. As shown inFIG. 23, by holding, in a play list, a table on which audio data piecescorresponding to the sub clips are written, the corresponding audio datapieces can be switched according to the switching of the sub clips.

Note that instead of the above-stated method of increasing the realisticsensation and sense of localization of sound emitted from the objectdisplayed in a 3D manner by preparing audio data pieces corresponding tothe respective sub clips, the following alternate method of localizingthe audio pieces on the corresponding sub clips is possible: (i) soundthat does not contribute to the realistic sensation of the 3D graphics(e.g. background sound) is stored, in an AV clip, as a first audiostream, (ii) only another sound that is emitted from a specific objector a specific character in a screen is stored, in the AV clip, as asecond audio stream, and (iii) the first audio stream and the secondaudio stream are played back simultaneously, while being mixed. Thiscontributes to reduction of data amount when a plurality of audiostreams are stored since audio compressing method having highercompression rate can be used by storing, as the second audio stream,comparatively simple sound such as conversation, hum of flies and thelike.

(1) In the embodiment 1 described in the above, the AV clip has an entrymap relating to the left-eye video stream, and the positions of therespective GOPs of the left-eye video stream precede the positions ofcorresponding GOPs of the right-eye video stream. However, the AV clipmay have an entry map relating to the right-eye video stream, and thepositions of the respective GOPs of the right-eye video stream mayprecede the positions of corresponding GOPs of the left-eye videostream.

(2) In the embodiment 1 described in the above, the playback apparatusincludes the left-eye video plane and the right-eye video plane.However, the playback apparatus may have only one video plane. In thiscase, the playback apparatus writes the left-eye pictures and right-eyepictures alternately in the video plane.

Also, although the playback apparatus has one video decoder, theplayback apparatus may have two video decoders; one for the left eye andthe other for the right eye. In this case, the demultiplexer outputs PESpackets to either the video decoder for the left eye or the videodecoder for the right eye, based on the PIDs included in the TS packets.

(3) In the embodiment 1 described in the above, each PTS for theright-eye picture is equal to PTS (for the left eye)+1/(the number offrames per second×2) with respect to a corresponding left-eye pictureshown by a certain time (PTS). However, the PTSs of the right eyepictures can be set at any interval as long as each of the PTSs of theright eye pictures is between a corresponding left-eye picture and thenext left eye picture.

(4) In the embodiment 1 described in the above, the AV clip has entrymaps relating to the left-eye video stream. However, the AV clip mayhave an entry map of the left-eye video stream and an entry map of theright-eye video stream. In table information of the right-eye entrymaps, a PTS is set for each entry point. Here, each of the PTSs is avalue obtained by adding, to a display delay ( 1/48), a PTS specified byan entry point of the left-eye video stream of an entry map in the AVclip.

When the playback start point is specified by time, the playback controlunit 2800 starts playback from an address preceding other addresseswhich is shown by address information corresponding to the time. Thismakes it possible to read GOPs starting from a head of the left-eye GOPand a head of the right-eye GOP of a specified time.

When the playback apparatus has extra memory, and each entry map can beread to memory, the left-eye video stream and the right-eye video streammay be multiplexed regardless of a storage positional relationship.

(5) In the embodiment 1 described in the above, between a header packetof the left-eye GOP and an end packet of the left-eye GOP is arrangedthe right-eye GOP corresponding to the left-eye GOP. However, theright-eye GOP may be arranged between the header packet of the left-eyeGOP and a header packet of the next left-eye GOP, or arranged before anend packet of the next left-eye GOP. By putting a restriction on aposition at which the left-eye GOP and the right-eye GOP correspondingto the left-eye GOP are multiplexed in the above-stated way, it can beensured that timing of reading the left-eye GOP and timing of readingthe right-eye GOP are prevented from becoming out of synchronizationwith to each other when random access to the AV data is performed.

(6) The embodiment 1 described in the above describes the displaycapable of displaying 3D graphics. However, a display capable ofdisplaying only 2D graphics exists in fact. In that case, it isdifficult for the user to judge whether or not the display is capable ofplaying back 3D graphics. Therefore, it is preferable that the playbackapparatus judges whether or not the display is capable of playing back3D graphics without the user even noticing it.

Specifically, this can be realized by the following method, for example.FIG. 24 is a flowchart showing inquiry processing that inquires to adisplay whether it is possible to display 3D graphics. The playbackapparatus and the display are connected to one another via a HDMI cable.Firstly, the playback apparatus inquires to the display whether it ispossible to display 3D graphics (Step S201). Receiving a response tosuch inquiry from the display (Step S202: Yes), the playback apparatusjudges whether or not the response shows that it is possible to display3D graphics (Step S203).

When the response shows that it is possible to display 3D graphics (StepS203: Yes), the playback apparatus sets a 3D display flag as valid witha predetermined memory of the playback apparatus (Step S204). Here, the3D display flag shows that the display is capable of displaying 3Dgraphics. When the response shows that it is not possible to display 3Dgraphics (Step S203: No) or no appropriate response is made (Step S202:No), the playback apparatus sets the 3D display flag as invalid (StepS205).

When the 3D flag can be accessed from the BD program, whether or not thedisplay connected to the playback apparatus is capable of displaying 3Dgraphics can be judged by the BD program. This allows controls to bemade, such as playing back a 3D title when the 3D display flag is valid,and playing back a 2D title when the 3D display flag is invalid.

Also, when the 3D flag is valid, a header and the like may includeinformation showing that content is 3D content when the playbackapparatus transmits the 3D content to the display. This makes itpossible, when the display receives 3D content, to reduce or lightencertain processing, such as by not performing the edge enhancement inthe display as described in the above, or to add appropriate processingfor viewing 3D graphics.

(7) In the embodiment 1 described in the above, a description is giventaking the successive separation method and polarized glasses method asexamples. However, when the stereoscopic viewing is realized using thelenticular lens, (i) setting is made such that the pictures of theleft-eye video stream and corresponding pictures of the right-eye videostream have the same PTSs, (ii) the left-eye pictures and thecorresponding right-eye pictures are arranged alternatively in alongitudinal direction on a screen simultaneously, and (iii) thelenticular lens is attached to the surface of the display. This allowsthe user to recognize the graphics displayed on the displaystereoscopically.

(8) Play lists for the 3D playback may be played back by a BD-Japplication. The BD-J application is a Java application that runs a BD-JTitle as a life cycle in a program execution unit that is fully providedwith Java 2 Micro_Edition (J2ME), Personal Basis Profile (PBP 1.0) andGlobally Executable MHP specification (GEM 1.0.2) for package mediatargets.

The program execution unit is composed of a Java™ (registered trademark)virtual machine, a configuration and a profile. The BD-J application canstart playback of AV by instructing the Java™ virtual machine togenerate a JMF player instance that plays back the play listinformation. The JMF (Java Media Frame work) player instance is actualdata generated on a heap memory of the virtual machine based on the JMFplayer class.

Also, the program execution unit includes a standard Java library fordisplaying JFIF (JPEG), PNG and other image data. Therefore, the BD-Japplication can realize a GUI framework. The GUI framework in the Javaapplication includes a HAVi framework stipulated in GEM 1.0.2, and amemory control navigation system in the GEM 1.0.2.

Thus, the BD-J application can realize screen display in which buttondisplay, text display, and online display (content of BBS) that arebased on the HAVi framework are combined with display of movingpictures. Also, the BD-J application can perform operation for suchscreen display using a remote control.

It is also possible to provide a combination of any of the embodimentsand supplementary remarks described in the above.

Although the present invention has been fully described by way ofexamples with reference to accompanying drawings, it is to be noted thatvarious changes and modifications will be apparent to those skilled inthe art. Therefore, unless otherwise such changes and modificationsdepart from the scope of the present invention, they should beconstructed as being included therein.

INDUSTRIAL APPLICABILITY

The present invention can be widely applied to any recording media thatstore therein 3D graphics.

1. A recording medium comprising: a digital stream area in which adigital stream is recorded, the digital stream including a plurality oftemporally-arranged GOP pairs; and a map information area in which mapinformation is recorded, the map information indicating entry addressesin one to one correspondence with entry times on a time axis of thedigital stream, each of the entry addresses showing a beginning of acorresponding one of GOP pair regions in the digital stream area,wherein each of the GOP pairs is composed of a first-type GOP and asecond-type GOP, each first-type GOP is data indicating a set of plainview pictures to be played back from a corresponding one of the entrytimes, and each second-type GOP is data to be played back together witha corresponding one of the first-type GOPs to provide a user with astereoscopic view of the digital stream, the data indicating adifference between a set of stereoscopic pictures and the set of plainview pictures.
 2. The recording medium of claim 1, wherein each of thefirst-type GOPs and each of the second-type GOPs are divided into afirst-type set of packets and a second-type set of packets,respectively, the first-type sets of packets and the second-type sets ofpackets are multiplexed together, all of the packets being assigned withconsecutive packet numbers according to an order of multiplexing, aheader packet from among each of the first-type sets of packetsprecedes, on the digital stream, a header packet from among acorresponding one of the second-type sets of packets, and each of theentry addresses in the map information is represented as one of thepacket numbers assigned to a header packet from among a correspondingone of the first-type sets of packets.
 3. The recording medium of claim2, wherein each of the second-type sets of packets divided from acorresponding one of the second-type GOPs is located before a next oneof the entry addresses that is immediately next to one of the entryaddresses that relates to a corresponding one of the first-type GOPs. 4.The recording medium of claim 2, wherein each of the second-type sets ofpackets divided from a corresponding one of the second-type GOPsprecedes an end packet of a corresponding one of the first-type GOPs. 5.The recording medium of claim 1, wherein pictures of each of thefirst-type GOPs are in one to one correspondence with pictures of eachof the second-type GOPs, and a display time of a certain picture of eachof the second-type GOPs is a value obtained by adding a display time ofa corresponding one of the pictures of a corresponding one of thefirst-type GOPs to a value obtained by dividing 1 by a frame rate of thecorresponding one of the first-type GOPs multiplied by
 2. 6. A recordingmedium comprising: a generating unit operable to generate a digitalstream, and a recording unit operable to record the generated digitalstream in the recording medium, wherein the digital stream includes (i)at least one first-type GOP which is data indicating a set of plain viewpictures to start being played back at an at least one entry time on atime axis of the digital stream, (ii) at least one second-type GOP whichis data to be played back together with the at least one correspondingfirst-type GOP to provide a user with a stereoscopic view of the digitalstream, the data indicating a set of differential pictures between a setof stereoscopic pictures and the set of plain view pictures, and (iii)map information indicating an at least one entry address in one to onecorrespondence with the at least one entry time in a recording area ofthe digital stream, and the generating unit divides the at least onefirst-type GOP and the at least one second-type GOP into a plurality ofpackets, and multiplexes together the plurality of packets such that aheader packet from among packets divided from the at least onefirst-type GOP precedes, on the digital stream, a header packet fromamong packets divided from the at least one corresponding second-typeGOP.
 7. A playback apparatus for playing back a digital stream in whichat least one first-type GOP and at least one second-type GOP are dividedinto a plurality of packets and are multiplexed together, the at leastone first-type GOP being data indicating a set of plain view pictures tostart being played back at an at least one entry time on a time axis ofthe digital stream, and the at least one second-type GOP being data tobe played back together with the at least one first-type GOP to providea user with a stereoscopic view of the digital stream, the dataindicating a set of differential pictures between a set of stereoscopicpictures and the set of plain view pictures, the playback apparatuscomprising: a video decoder operable to decode the at least onefirst-type GOP and the at least one second-type GOP to obtain movingpictures; a first video plane that stores moving pictures obtained bydecoding the at least one first-type GOP; a second video plane thatstores moving pictures obtained by decoding the at least one second-typeGOP; a display unit; and an output unit operable to output, to thedisplay unit, the moving pictures stored in the first video plane andthe moving pictures stored in the second video plane alternately,wherein each of the packets includes a packet ID showing whether dataincluded in the packet relates to the at least one first-type GOP or theat least one second-type GOP, and the video decoder outputs the decodedmoving pictures to: (i) the first video plane when the packet ID showsthat the data relates to the at least one first-type GOP; and (ii) thesecond video plane when the packet ID shows that the data relates to theat least one second-type GOP.
 8. A playback method that plays back adigital stream in which at least one first-type GOP and at least onesecond-type GOP are divided into a plurality of packets and aremultiplexed together, the at least one first-type GOP being dataindicating a set of plain view pictures to start being played back at anat least one entry time on a time axis of the digital stream, and the atleast one second-type GOP being data to be played back together with theat least one first-type GOP to provide a user with a stereoscopic viewof the digital stream, the data indicating a difference between a set ofstereoscopic pictures and the set of plain view pictures, the methodcomprising the steps of: decoding the at least one first-type GOP andthe at least one second-type GOP to obtain moving pictures; andoutputting, to a display device, moving pictures obtained by decodingthe at least one first-type GOP in a first video plane, and movingpictures obtained by decoding the at least one second-type GOP in asecond video plane alternately, wherein each of the packets includes apacket ID showing whether data included in the packet relates to the atleast one first-type GOP or the at least one second-type GOP, and thevideo decoder outputs the decoded moving pictures to: (i) the firstvideo plane when the packet ID shows that the data relates to the atleast one first-type GOP; and (ii) the second video plane when thepacket ID shows that the data relates to the at least one second-typeGOP.